Composition of Web Services Using Markov Decision Processes and Dynamic Programming

We propose a Markov decision process model for solving the Web service composition (WSC) problem. Iterative policy evaluation, value iteration, and policy iteration algorithms are used to experimentally validate our approach, with artificial and real data. The experimental results show the reliability of the model and the methods employed, with policy iteration being the best one in terms of the minimum number of iterations needed to estimate an optimal policy, with the highest Quality of Service attributes. Our experimental work shows how the solution of a WSC problem involving a set of 100,000 individual Web services and where a valid composition requiring the selection of 1,000 services from the available set can be computed in the worst case in less than 200 seconds, using an Intel Core i5 computer with 6 GB RAM. Moreover, a real WSC problem involving only 7 individual Web services requires less than 0.08 seconds, using the same computational power. Finally, a comparison with two popular reinforcement learning algorithms, sarsa and Q-learning, shows that these algorithms require one or two orders of magnitude and more time than policy iteration, iterative policy evaluation, and value iteration to handle WSC problems of the same complexity

Chagas Parasite Detection in Blood Images Using AdaBoost

The Chagas disease is a potentially life-threatening illness caused by the protozoan parasite, Trypanosoma cruzi. Visual detection of such parasite through microscopic inspection is a tedious and time-consuming task. In this paper, we provide an AdaBoost learning solution to the task of Chagas parasite detection in blood images. We give details of the algorithm and our experimental setup. With this method, we get 100% and 93.25% of sensitivity and specificity, respectively. A ROC comparison with the method most commonly used for the detection of malaria parasites based on support vector machines (SVM) is also provided. Our experimental work shows mainly two things: (1) Chagas parasites can be detected automatically using machine learning methods with high accuracy and (2) AdaBoost + SVM provides better overall detection performance than AdaBoost or SVMs alone. Such results are the best ones known so far for the problem of automatic detection of Chagas parasites through the use of machine learning, computer vision, and image processing methods

A Multiagent Architecture for Concurrent Reinforcement Learning ∗

In this paper we propose a multiagent architecture for implementing concurrent reinforcement learning, an approach where several agents, sharing the same environment, perceptions and actions, work towards one only objective: learning a single value function. We present encouraging experimental results derived from the initial phase of our research on the combination of concurrent reinforcement learning and learning from demonstration.

Alineando un modelo mediano GPT en inglés a un dominio cerrado y pequeño en español

In this paper, we propose a methodology to align a medium-sized GPT model, originally trained in English for an open domain, to a small closed domain in Spanish. The application for which the model is finely tuned is the question answering task. To achieve this we also needed to train and implement another neural network (which we called the reward model) that could score and determine whether an answer is appropriate for a given question. This component served to improve the decoding and generation of the answers of the system. Numerical metrics such as BLEU and perplexity were used to evaluate the model, and human judgment was also used to compare the decoding technique with others. Finally, the results favored the proposed method, and it was determined that it is feasible to use a reward model to align the generation of responses.En este artículo se propone una metodología para alinear un modelo GPT de tamaño mediano, entrenado originalmente en inglés y de dominio abierto, a un dominio cerrado y pequeño en español. La aplicación para la cual se afina el modelo es para una tarea de preguntas y respuestas. Para lograr este objetivo también fue necesario entrenar e implementar otra red neuronal (a la cual llamamos modelo de recompensas) que pudiera calificar y determinar si una respuesta es adecuada para una determinada pregunta. Este componente sirvió para mejorar la decodificación y generación de las respuestas del sistema. Para la evaluación del modelo se utilizaron métricas numéricas como BLEU y perplejidad, y también se utilizó la evaluación a juicio humano, comparando la técnica de decodificación con otras. Finalmente, los resultados favorecieron el método propuesto, y se determinó que es factible utilizar un modelo de recompensas para alinear la generación de respuestas